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The  Standardization  of  the  C  Language 


Lawrence  Rosier 
AT&T  Bell  Laboratories 
Summit,  New  Jersey 


Workshop  Abstract 

The  American  National  Standards  Institute  (ANSI)  Technical  Committee  for  the 
Programming  Language  C  (X3J11)  has  been  meeting  quarterly  since  June,  1983.  The 
Committee  is  composed  of  more  than  30  representatives  of  the  C  community,  including 
vendors  of  compilers  and  applications,  consultants  and  educators.  The  implementations 
considered  run  under  the  UNIX1  System,  under  other  systems,  or  stand-alone. 

The  goal  of  the  Committee  is  to  produce  a  draft  standard  for  public  comment  early  in 
1985.  Work  is  proceeding  in  three  areas:  environment,  library,  and  language.  The 
environment  activity  is  attempting  to  standardize  the  characteristics  of  the  compilation 
and  execution  environments,  including  the  attributes  of  external  files.  The  library  activity 
is  adapting  the  proposed  /usr/group  standard  to  the  requirements  of  execution 
environments  that  are  not  UNIX-like.  This  session  will  focus  on  the  language  activity. 

The  base  document  for  the  language  proper  is  derived  from  the  C  Reference  Manual  that 
is  Appendix  A  of  the  1978  book  by  B.W.  Kernighan  and  D  M.  Ritchie  (K&R).  The  style  and 
approach  are  fundamentally  unchanged.  Some  copies  of  the  most  recent  working  draft 
will  be  available. 

The  proposed  changes  from  K&R  will  be  presented.  Considerable  time  will  be  available 
for  detailed  discussion.  Comments  and  suggestions  from  the  audience  will  be 
appreciated. 
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A  Simple  Simulation  Toolkit  in  C 

Robert  P.  Warnock,  III 

Fortune  Systems  Corporation 
Redwood  City,  California  94061 

Bakul  Shah 

458  Feme 

Palo  Alto,  CA  94306 
ABSTRACT 


A  simple  set  of  simulation  tools  is  presented  which  offers 
much  of  the  power  of  specialized  simulation  languages  such 
as  GPSS  or  SIMSCRIPT,  while  not  interfering  with  the 
C  programmer's  native  style.  The  toolkit,  a  library  of 
C  routines,  comprises  three  conceptual  layers  (the  user's 
simulation  is  the  fourth),  each  reflecting  some  well-known 
design  borrowed  from  other  systems.  From  the  "bottom"  up, 
they  are:  (1)  Co-routines  and  process  objects;  (2)  Condition 
variables  and  priority  queues;  and,  (3)  A  simulation  kernel 
(the  "clock"  object  and  the  "busy"  statement).  Co-routines 
and  process  objects  are  as  in  the  BLISS  language.  Condition 
variables,  priority  queues,  and  the  simulation  kernel  itself 
are  from  Holt's  Concurrent  Euclid.  The  overall  idea  of  pro¬ 
viding  a  simple  skeleton  for  user-written  simulations  is 
from  CMU's  P00MAS  package  (POOr  MAn's  Simula). 

Two  examples  of  use  are  given,  with  different  styles  of  out¬ 
put.  One  produces  a  discrete  "event  log"  report,  and  the 
other  a  quasi-continuous  "logic  analyzer"  display. 

It  is  hoped  that  this  paper  will  encourage  programmers  and 
engineers  to  make  more  use  of  simulations  in  their  designs, 
as  they  realize  that  there  is  no  particular  magic  involved. 

Introduction 


From  time  to  time,  the  professional  programmer  or  engineer  encounters 
design  situations  which  may  be  too  complex  to  handle  easily  with 
numerical  or  algebraic  analysis.  This  difficulty  may  be  due  to 
theoretical  intractability  of  the  subject  area,  or  it  may  be  simply 
that  the  area  is  new  to  the  designer,  who  does  not  yet  have  a  reliable 
intuitive  model  to  lean  on  in  deriving  his  or  her  analysis.  In  either 
case,  discrete  simulation  is  an  often  useful  tool  to  explore  the 
design  space  and  to  gain  some  feeling  for  the  magnitudes  of  the 
interactions  between  variables.  In  extreme  cases,  a  simulation  is  the 
only  way  to  get  usable  numerical  results.  Even  when  acceptably  accu¬ 
rate  analytic  results  are  available  for  overall  system  performance, 
simulation  may  reveal  some  surprising  interactions  which  were  unanti¬ 
cipated  by  analysis.  (An  example  is  catastrophic  buffer  congestion  in 
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packet  networks.) 

Simulation  and  analysis  therefore  can  complement  each  other  in  refin¬ 
ing  a  design.  However,  to  many  programmers  and  engineers,  the  term 
"simulation"  conjures  visions  of  large  mainframes  spending  hours  of 
CPU  time,  arcane  unapproachable  special-purpose  languages,  and  many 
heroic  hours  of  data  and  control-file  preparation.  For  many  of  the 
simulation  needs  encountered  in  the  microprocessor  and  minicomputer 
worlds,  doing  useful  simulations  need  not  involve  leaving  ones  normal 
computing  environment. 

What  is  needed  is  a  view  of  simulations  as  a  normal  part  of  ones  per¬ 
sonal  skillset.  Working  through  some  introductory  examples  often 
helps  one  acquire  this  view,  so  after  presenting  the  simulation 
toolkit,  we  sketch  a  modest  DMA/memory-bus  simulation  to  illustrate 
the  major  points.  While  quite  simple,  this  simulation  was  useful  in 
discovering  a  bus-arbitration  problem  in  a  commercial  computer.  The 
original  version  of  the  simulation  toolkit  and  the  DMA/bus  simulation 
were  developed  over  one  "long  weekend"  while  investigating  this  prob¬ 
lem.  It  is  hoped  that  this  paper  will  encourage  other  programmers  and 
engineers  to  make  use  of  simulations  in  their  designs,  as  they  realize 
that  there  is  no  particular  magic  involved. 

Overview  of  the  Simulation  Toolkit 

We  describe  a  simple  set  of  simulation  tools  which  offer  much  of  the 
power  of  specialized  simulation  languages  such  as  GPSS  or  SIMSCRIPT, 
while  not  interfering  with  the  C  programmer's  native  style.  In  addi¬ 
tion,  it  allows  some  experimentation  with  object-oriented  programming 
techniques.  Its  features  include: 

Processes:  Dynamic  creation  and  destruction  of  any  number  of  process 
objects  or  types;  No  process  object  semantics  are  assumed, 
only  the  bare  framework  is  provided;  Process  objects  may  be 
passed  parameters  at  creation  time;  Processes  can  call  any 
normal  C  library  subroutines  (such  as  "printf"). 

Efficient:  Reasonably  low  simulation  overhead  (medium-sized  simula¬ 
tions  run  at  human  reading  speed),  process-switch  time  is 
comparable  to  subroutine  call/return  times. 

Portable:  Machine  independent  above  the  bottom  layer  (implementations 
for  the  bottom  layer  are  available  for  the  Motorola  68000 
and  the  VAX  11/780). 

Open:  Any  portion  of  the  toolkit  may  be  replaced  or  modified  to 

fit  a  particular  application.  As  discussed  below,  no  out¬ 
put  procedures  are  supplied.  The  user  has  complete  control 
of  the  display. 

The  overall  idea  of  providing  a  simple  skeleton  for  user-written  simu¬ 
lations  is  from  the  POOMAS  package  (Poor  Man's  Simula)  [DECUS]  . 
Written  in  the  BLISS  programming  language  (a  goto-less  type-less 
language  of  the  ALGOL  and  BCPL  traditions)  [WULF71],  POOMAS  was  a 
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package  of  macros  and  library  routines  which  allowed  one  to  write 
BLISS  programs  in  the  style  of  SIMULA  simulations  [DAHL72b]  [PALME73]. 

The  approach  discussed  here,  a  library  of  C  routines,  is  presented  as 
three  conceptual  layers,  while  the  user's  application  forms  the  fourth 
and  fifth.  Each  layer  reflects  some  well-known  design  borrowed  from 
other  systems.  From  the  "bottom"  up,  they  are: 

1.  Co-routines  and  process  objects; 

2.  Condition  variables  and  priority  queues; 

3.  A  simulation  kernel  (the  "clock"  object  and  the  "busy"  state¬ 
ment). 

To  create  a  complete  simulation,  the  user  supplies: 

*1.  The  output  display  and  monitoring  procedures;  and 

5.  The  procedures  which  model  the  simulated  objects  themselves. 

Layer  U  Co-routines  and  process  objects 

In  order  to  simulate  the  behaviour  of  many  objects  acting  "simultane¬ 
ously",  we  model  each  object  as  an  independent  process.  Processes 
communicate  via  global  variables  or  pointers  to  shared  data  objects. 
For  efficiency,  all  of  the  simulation  processes  are  sub-processes  of 
one  timesharing  process  or  job  [1],  effectively  providing  multipro¬ 
gramming  within  the  job.  The  lowest  level  of  the  toolkit  is  therefore 
a  process  scheduler,  which  allows  individual  sub-processes  to  be 
suspended  and  resumed  later.  This  can  be  easily  done  in  any  program¬ 
ming  language  which  provides  a  mechanism  for  coroutines. 

Coroutines  [KNUTH73]  [DAHL72a]  are  a  generalization  of  the  more  fami¬ 
liar  subroutine  mechanism,  in  that  when  one  coroutine  A  "returns  to" 
or  "calls"  another  coroutine  B,  the  location  of  the  call  in  A  is 
remembered.  When  A  is  called  again  (by  B  or  other  coroutines!,  it 
resumes  from  where  it  left  off,  rather  than  starting  over  at  the  top. 
Where  coroutines  are  not  supported  by  standard  software,  they  can 
often  be  provided  by  a  small  assembly-language  routine  [2]. 

The  co-routine  and  process  object  models  used  here  are  derived  from 

[1]  To  simplify  terminology,  from  here  on  "process" 
will  mean  a  sub-process  within  one  timesharing  process, 
job,  or  program.  Note  that  a  "job"  is  a  single  UNIX 
user  process. 

[2]  The  C  library  routines  "setjmp(3)"  and  "longjmp(3)" 
may  be  used  to  implement  coroutines.  We  chose  to  pro¬ 
vide  a  single  assembly-language  "resume",  for  clarity. 

See  also  "ctxsw.s",  in  [COMER84]. 
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the  exchj  (exchange  jump)  and  create  constructs  in  the  BLISS- 10 
language  [DEC731*  A  process  object  or  process  is  defined  by  a  data 
structure  which  contains  "static"  variables  local  to  the  process,  a 
stack  of  subroutine  call  frames,  and  the  process  context  (PC,  stack 
pointer,  registers)  when  not  active.  To  the  C  compiler  or  the  operat¬ 
ing  system,  a  process  object  is  merely  another  user  data  space  object 
allocated  either  statically  from  a  fixed  array  or  dynamically  from  the 
heap  via  "malloc". 

At  any  one  time,  exactly  one  process  is  running,  and  the  global  vari¬ 
able  co  currentproc  holds  a  pointer  to  it.  The  state  of  the  current 
process  may  be  saved  and  another  process  activated  with  the  call: 

co_resume(other_proc) ; 

This  is  a  very  low-level  mechanism,  and  does  not  save  the  address  of 
the  current  process  object,  as  "co_currentproc"  is  overwritten  with  a 
pointer  to  the  new  "other_proc".  The  caller  must  arrange  for  the 
current  process  to  be  enqueued  (or  otherwise  remembered)  for  subse¬ 
quent  activation,  or  the  "co_resume"  call  can  never  return.  (Nor¬ 
mally,  the  caller  is  one  of  the  Layer  2  or  Layer  3  routines  which  per¬ 
form  exactly  this  function.) 

But  before  a  process  can  be  run  with  "co_resume",  a  process  object 
must  be  created  for  it,  and  the  object  must  be  filled  in  with  a  valid 
stack  frame,  address  of  the  code  for  the  process,  etc. [3]  This  is  done 
by  allocating  space  [4]  for  the  process  object,  then  calling 
co  create: 

proc  =  co_create(address,  size,  handler, 

function,  argl,  arg2,  ...,  argN); 

This  creates  a  process  object  in  the  space  given  by  address  and  size. 

Function  is  the  code  which  gets  executed  when  the  process  is  resumed, 
and  is  an  ordinary  C  subroutine.  The  stack  frame  is  arranged  so  that 
on  the  first  resume  of  the  process,  the  function  is  entered  from  the 
beginning  exactly  as  if  it  had  been  called  with  a  normal  subroutine 
call  with  arguments  argl  ...  argN.  The  return  value  of  "co_create"  is 
a  pointer  to  the  process  object,  for  use  in  later  calls  such  as 
"co_resume". 

If  the  function  should  exit  via  a  normal  return,  handler  is  called  (by 
the  internal  routine  co  exit  )  with  arguments  of  the  address  of  the 
process  object,  its  size,  and  the  return  value  of  "function".  The 
exit  handler  should  perform  whatever  cleanup  is  needed  (such  as  de- 

[3]  A  small  dummy  process  object  must  also  be  created 
and  initialized  for  the  main  routine  before  the  first 
"co_resume"  in  the  program. 

[4]  Choosing  the  size  may  be  tricky.  See  Appendix  A. 
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allocating  the  space  for  the  process  object),  and  resume  some  other 
process.  It  is  a  fatal  error  for  the  exit  handler  to  return. 

Layer  2:  Condition  Variables  and  Priority  Queues 

After  arranging  for  many  processes  and  a  way  to  switch  between  them, 
the  next  thing  needed  is  a  method  of  synchronizing  their  executions. 
For  that,  we  use  condition  variables  and  priority  queues  as  in  Con¬ 
current  Euclid  [HOLT83],  however  "monitors"  per  se  are  not  provided. 
Rather,  the  condition  queues  are  used  directly  by  the  user  for  inter¬ 
process  synchronization  [5]. 

A  condition  variable  is  a  queue  of  processes  waiting  for  the  condition 
associated  with  that  variable  to  become  true.  A  process  which  exe¬ 
cutes  the  call 


q_wait(cond) 

waits  on  the  condition  queue  for  cond  (which  must  have  been  declared 
q  var ) .  A  later  signal  call  will  remove  the  first  such  waiting  pro¬ 
cess.  By  giving  a  second  argument, 

q_wait(cond,  prio) 

the  process  waits  on  the  condition  queue  for  cond  with  priority  prio 
(lower  integer  values  are  higher  priorities) .  A  later  signal  will 
remove  the  process  with  highest  priority. 

q_signal(cond) 

If  there  is  a  process  waiting  on  cond,  the  signal  puts  the  current 
process  onto  the  ready  queue,  and  resumes  the  highest  priority  process 
waiting  on  the  condition.  If  there  are  no  processes  currently  wait¬ 
ing,  it  simply  selects  the  next  process  from  the  ready  queue  and 
resumes  it  (possibly  the  same  one  which  called  "signal"). 


Layer  3:  Simulation  kernel 

Once  you  can  switch  between  processes  and  can  synchronize  them  with 
condition  queues,  the  rest  is  easy.  The  basic  function  which  turns 
all  of  the  above  mechanism  into  a  simulation  package  is  the  busy 
statement. 

sim_busy(delta_time) 

This  puts  the  current  process  on  the  sleep  queue  so  that  it  will  run 

[5]  On  a  single-processor  system  without  interrupts, 
this  is  not  a  limitation.  True  monitors  can  be  impl- 
mented,  if  needed,  by  adding  the  routines  "mon_enter ( )" 
and  "mon  exit()". 
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after  delta  time  units  of  simulated  time  have  elapsed,  and  selects 
another  process  for  running. 

When  there  are  no  available  processes  to  run  on  the  ready  queue,  simu¬ 
lated  time  (the  global  clock  variable  sim  time  )  is  advanced  to  the 
time  of  the  next  event  on  the  sleep  queue,  and  that  process  is 
dequeued  and  run.  This  has  the  following  implications: 

1.  Simulated  time  advances  in  "leaps”,  skipping  over  periods  during 
which  nothing  happens.  This  helps  to  make  the  simulation  reason¬ 
ably  efficient. 

2.  No  matter  how  much  real  CPU  time  a  process  takes,  it  will  appear 
infinitely  fast  in  the  simulation.  The  "busy"  statement  and  the 
"wait"  are  the  only  ways  simulated  time  can  pass. 

Finally,  to  get  a  process  going  in  the  beginning,  there  is  the  start 
statement. 

sim_start(proc) 

This  starts  process  proc  immediately  by  placing  it  on  the  ready  queue 
(although  it  will  not  actually  run  until  the  current  process  and  all 
other  ready  processes  have  executed  "busy"  or  "wait"  statements). 

Writing  Simulations  Using  the  Toolkit 

To  construct  a  complete  simulation,  the  programmer  must  add  two  more 
"layers".  The  first  is  the  output  display  routine  or  process,  and  the 
second  comprises  the  actual  routines  to  define  the  behavior  of  the 
objects  in  the  simulation.  While  conceptually  the  output  function  is 
a  "lower"  level  than  the  simulation  itself,  in  fact  they  must  be 
designed  together.  The  selection  of  what  portion  of  the  state  of  the 
simulation  to  display  and  how  to  format  it  is  very  dependent  on  the 
application  domain  and  on  the  properties  of  the  simulated  system. 

Simulations  are  usually  concerned  with  a  variable  number  of  objects 
which  arrive  (appear),  spend  most  of  their  lives  waiting  in  one  or 
more  queues  to  get  service,  get  some  service  or  other  (possibly  loop¬ 
ing  in  the  system),  and  eventually  depart  (disappear).  Sometimes  the 
number  of  objects  is  fixed,  and  what  is  being  simulated  is  various 
state  changes  within  the  objects  as  they  receive  simulated  input  from 
outside  the  system  and  interact  with  one  another. 

In  most  cases,  however,  the  information  desired  from  the  simulation 
will  fall  into  one  of  two  general  styles,  which  we  will  call 
event-oriented  or  time-oriented  depending  on  the  style  of  the  output 
display.  Oddly  enough,  the  output  style  seems  to  have  little  to  do 
with  whether  the  system  has  a  variable  number  of  objects  or  a  fixed 
number,  but  whether  one  is  focussing  on  the  fine-grain  sequential 
behavior  or  the  overall  parallel  behavior  of  the  system. 

The  event-oriented  output  display  shows  each  of  the  major  transitions 
of  each  object,  queue,  or  server,  like  this  example: 
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Time(ns):  Device:  Event 


1:  clock:  signal  busFree 
1:  CPU:  bus  acquired 
10:  ENET1:  next  word  ready 

10:  ENET2:  next  word  ready 

10:  DISK:  next  word  ready 
1610:  ENET1:  next  word  ready 

1610:  ENET2 :  next  word  ready 

1621:  CPU:  bus  released 
1621:  REFRESH:  bus  acquired 
216 1:  REFRESH:  bus  released 
2 1 6 1 :  ENET1:  bus  acquired 
2701:  ENET1:  word  transferred 
3210:  DISK:  next  word  ready 
3210:  ENET1:  next  word  ready 

3210:  ENET2:  next  word  ready 

3241:  ENET1:  word  transferred 
3781:  ENET1:  word  transferred 
3781:  ENET1 :  bus  released 
3781:  ENET2:  bus  acquired 
4321 :  ENET2:  word  transferred 

This  mode  is  generally  implemented  by  display  calls  inside  each  queue 
or  server  object.  These  may  be  as  simple  as  a  "printfO",  but  for 
ease  of  modification  it  is  more  usual  to  have  a  common  print  routine, 
which  may  choose  to  output  or  not  depending  on  some  global  printing- 
level  parameter  (much  like  "DEBUGO"  lines  used  in  many  C  programs). 

When  one  is  more  interested  in  the  overall  state  of  the  system,  such 
as  the  size  of  queues  or  the  state  of  several  objects  simultaneously, 
it  is  often  convenient  to  view  time  as  quasi-continuous.  The  time- 
oriented  or  "logic  analyzer"  style  of  display  shows  a  fixed  amount  of 
time  per  line  of  output,  and  each  line  outputs  the  status  of  a  fixed 
number  of  objects  (like  the  "channels"  of  a  logic  analyzer).  An  exam¬ 
ple: 


Time(ns) 

BUS 

Refresh 

Enettfl 

Enet//2 

Disk  CPU 

Bus 

clock 

14590 

ENET1 

DMAread 

DataRdy 

up 

\\ 

15130 

ENET2 

DMAread 

up 

\\ 

15670 

CPU 

I /Oread 

up 

\\ 

16210 

CPU 

DataRdy 

DataRdy 

DataRdy  I /Oread 

up 

\\ 

16750 

CPU 

RefrReq  DataRdy 

DataRdy 

DataRdy  I /Oread 

up 

\\ 

17290 

REFRESH 

MemRefr 

DataRdy 

DataRdy 

DataRdy 

up 

\\ 

17830 

ENET1 

DMAread 

DataRdy 

DataRdy 

up 

\\ 

18370 

ENET1 

DMAread 

DataRdy 

DataRdy 

up 

\\ 

18910 

ENET2 

DMAread 

DataRdy 

up 

\\ 

19450 

ENET2 

DataRdy 

DMAread 

DataRdy 

up 

\\ 

19990 

ENET1 

DMAread 

<wait> 

DataRdy 

up 

\\ 

20530 

ENET2 

DMAread 

DataRdy 

up 

\\ 

21070 

ENET2 

DataRdy 

DMAread 

DataRdy 

up 

\\ 

21610 

ENET1 

DMAread 

DataRdy 

up 

\\ 

22150 

DISK 

DMAwrit 

up 

\\ 
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22690  ENET1  DMAread  DataRdy  <wait>  up  \\ 

23230  ENET2  DMAread  <wait>  up  \\ 

23770  DISK  DMA  writ  up  \\ 

For  this  mode  of  display,  the  objects  must  record  their  states  in  glo¬ 
bally  accessible  locations,  so  that  an  independent  display  process  can 
sample  those  states  at  regular  intervals  to  format  and  output  them. 
(To  truly  emulate  a  logic  analyzer,  one  may  even  keep  a  pair  of  "tran¬ 
sition  detectors"  for  each  state  variable,  as  "glitch  catchers",  so 
that  time  intervals  with  more  than  one  clean  transition  can  be  noted.) 
Time-oriented  displays  have  the  advantage  that  one  may  stretch  or 
compress  time  without  altering  the  simulated  objects,  merely  by  chang¬ 
ing  the  loop  time  in  the  display  process  (note  the  fine  detail  now 
showing  in  the  "bus  clock"): 


Time (ns)  BUS 

Refresh  Enet#1  Enet//2  Disk 

CPU 

Bus  clock 

1*1754  ENET1 

DMAread  DataRdy 

dn  1 

14784  ENET1 

DMAread  DataRdy 

up  \\ 

14814  ENET1 

DMAread  DataRdy 

up 

14844  ENET1 

DMAread  DataRdy 

up  1 

14874  ENET1 

DMAread  DataRdy 

dn  // 

14904  ENET1 

DMAread  DataRdy 

dn  | 

14934  ENET1 

DMAread  DataRdy 

dn  | 

14964  ENET2 

DMAread 

up  \\ 

14988  ENET2 

DMAread 

up  1 

The  displays  shown  above  were  output  from  two  versions  of  a 
DMA/memory-bus  simulation  used  to  investigate  interactions  among  dev¬ 
ices  generating  a  heavy  DMA  bus  load  in  a  commercial  microcomputer. 
The  devices  being  emulated  were  the  CPU  itself  (making  continuous 
references  to  slow  I/O  registers),  two  10  Mbit/sec  Ethernet  controll¬ 
ers,  a  5  Mbit/sec  disk  controller,  and  the  dynamic  RAM  refresh  pro¬ 
cess.  To  get  a  feeling  for  an  event-oriented  process,  consider  the 
routine  "refresh"  which  emulated  the  RAM  refresh  logic,  shown  in  its 
entirety  below: 

refresh(name,  busFree,  busCycle,  priority) 

char  *  name; 

q_var  busFree; 

int  busCycle,  priority; 

{ 

int  refreshPeriod  =  1*1580;  /*  ~2ms.  /  128  rows  */ 
for  (;;)  { 

wait(busFree,  priority);  /*  grab  the  bus  */ 

msg(name,  ":  bus  acquired\n" ) ;  /*  got  it  #/ 

busy( busCycle ) ;  /*  refresh  one  row  */ 

msg(name,  ":  bus  released\n" ) ; 

signal ( busFree ) ;  /*  let  go  */ 

busy(refreshPeriod) ;  /*  wait  a  bit  */ 

} 

} 
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Below  is  the  same  "for"  loop  for 
"refresh" : 

for  (;;)  { 

msg2(name,  "RefrReq"); 
wait(busFree,  priority); 
msg2 ( "BUSUSER" , name ) ; 
msg2(name, "MemRefr" ) ; 
busy(busCycle) ; 
msg2(" BUSUSER","  "); 

msg2(name,"  "); 

signal ( busFree ) ; 
busy(refreshPeriod) ; 

} 

While  it  is  possible  to  intermingle  both  styles  in  one  simulation,  it 
gets  quite  cumbersome,  and  is  not  recommended.  If  it  is  absolutely 
required  to  have  both,  a  more  sophisticated  display  package  can  sim¬ 
plify  the  appearance  of  the  process  routines,  but  inevitably  intro¬ 
duces  its  own  run-time  overhead. 

Summary 

We  have  outlined  a  simple  set  of  tools  which  provide  the  C  programmer 
the  ability  to  perform  discrete  event  simulations.  The  toolkit  uses 
coroutines,  condition  queues,  and  a  simulated  clock  to  provide  quasi¬ 
parallel  execution  of  multiple  sub-processes  in  a  single  user  program. 
An  example  was  sketched  of  an  application  of  this  toolkit  to  the 
analysis  of  a  computer's  memory  bus.  We  encourage  others  to  experi¬ 
ment  with  the  use  of  simulations  in  design  analysis  and  problem  solv¬ 
ing. 
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Appendix  A  -  Hints,  Restrictions,  and  Warnings 
Stack  Locations 

The  primary  obstacle  to  this  approach  is  hardware  and/or  software 
which  is  restrictive  about  where  user  stacks  may  be  located,  since  the 
the  process  objects  and  therefore  the  stacks  are  kept  in  user  data 
space.  It  is  possible  but  cumbersome  to  construct  a  dynamic  coroutine 
mechanism  in  which  the  stacks  live  within  the  job's  main  stack  space. 
Fortunately,  most  UNIX*  implementations  permit  stack  pointers  anywhere 
in  user  data  space. 

Stack  sizes 

The  user  is  responsible  for  determining  the  size  of  the  process 
object,  depending  on  the  number  of  "static"  variables  and  the  size  of 
the  stack  needed  for  the  process.  The  main  process  is  the  only  one 
for  which  system-provided  stack  growth  facilities  will  work,  so  if 
memory  space  is  scarce,  the  main  process  (once  it  has  started  the 
simulation)  may  best  be  used  as  a  service  process  to  handle  stack¬ 
intensive  requests  (such  as  "printf (3)" ) • 

Although  the  tuning  of  the  object  size  and  watching  for  stack-overflow 
problems  can  be  tricky  for  large  simulations,  in  fact  the  authors  have 
not  encountered  stack  problems  in  the  medium-sized  simulations  this 
toolkit  was  written  to  handle.  Each  process  was  simply  given  an  ample 
stack  (1000  bytes  or  so),  and  no  fine  adjustments  were  done. 

"Nargs" 

The  routine  "nargs"  is  very  machine  dependent,  and  cannot  be  written 
at  all  on  some  machine  or  compiler  implementations.  Its  main  use  here 
is  in  "createProc" ,  where  it  provides  a  naturalness  to  the  process 
procedure  invocation.  (A  lesser  use  is  in  the  "wait"  routine.)  The 
use  of  "nargs"  can  be  avoided  if  necessary  in  several  ways:  (1)  add  a 
parameter  to  "createProc"  to  explicitly  indicate  the  number  of  argu¬ 
ments,  (2)  force  all  processes  to  have  a  fixed  number  of  arguments, 
(3)  use  an  argument  block  pointer  (such  as  "argc"  and  "argv")  or  (4) 
place  the  arguments  into  the  process  object  at  process-creation  time. 


*UNIX  is  a  Trademark  of  Bell  Laboratories. 
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Relating  Benchmarks  to  Performance  Projections 

or 

What  do  you  do  with  20  pounds  of  benchmark  data? 

Gene  Dronek 

Aim  Technology 
Santa  Clara,  Ca  95051 
(498)  727-3711 


1.0  ABSTRACT 


UNIX  operating  systems  seem  to  invite  benchmarking 
measurements.  So  much  so,  UNIX  journals  now  include  benchmark- 
of-the-month  columns.  But  can  you  really  use  benchmarks  to 
justify  purchase  decisions,  or  is  benchmarking  merely  an 
interesting  exercise? 

At  Aim  Technology,  we  experimented  with  a  linear- 
combination  model  that  projects  performance  from  1  to  50  columns 
of  raw  benchmark  data.  Our  stated  goal  was  to  construct  a 
decision-making  tool  for  use  by  non-experts.  In  our 
implementation,  you  request  a  projection  by  specifying 
percentage  mixes  of  applications,  such  as  10X  word-processing 
plus  90X  compiling. 

At  first,  organizing  displays  of  benchmark  database  data  in 
reasonable  space  was  difficult.  We  tried  Kiviat  diagrams  and 
found  they  could  display  50  performance  factors  for  20  machines 
simultaneously  on  one  diagram.  Although  artistic  looking,  Kiviat 
diagrams  do  not  compare  as  easily  as  rectangular  plots. 
Ultimately,  Kiviat  diagramming  inspired  the  compact  displays  we 
call  VIEWS. 

In  the  paper,  we  describe  the  linear  model  in  detail,  with 
aspects  of  software  engineering.  We  define  the  "Checksummed 
Header"  puzzle  for  readers  to  enjoy  before  the  conference. 


1.1  PROJECT  GOALS 


Work  on  this  project  was  done  at  Aim  Technology  in  the  ten 
month  interval  from  August  1983  to  May  1984.  We  had  three 
stated  goals.  Primarily,  we  wanted  a  professional  tool  for 
computer-purchase  decision  making.  It  had  to  be  utterly 
portable,  and  it  was  to  be  used  by  non-experts. 

Secondarily,  we  wanted  a  tool  that  could  also  generate 
reports  suitible  for  publication.  Reports  had  to  be  field- 
configurable,  again  by  non-experts. 
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2.0  BENCHMARKING  PROCEDURE 


The  benchmarking  strategy  we  developed  separates  data 
acquisition  from  data  reporting.  An  ascii  database  connects 
the  two  phases.  This  arrangement  fosters  portability  and 
allowed  us  initially  to  experiment  with  several  data 
acquisition  methods,  yet  utilize  one  common  report 
generator.  Early  during  development,  we  used  twelve 
performance  measuring  programs,  all  feeding  into  the  one 
database.  Eventually,  all  twelve  codes  were  integrated  into 
one  quick  benchmark  code  —  dubbed  "Q-BENCH"  (for  Quick- 
Benchmark),  but  the  ascii  database  format  was  kept. 


Q  -  BENCH  * 


-> 

-> 

-> 


tixl 

>ix2 

>1x3 


SELECT  » 

>  * 
t  • 

•  •  •  •  * 


CRT 

LINEPRINTER 

REPORTS 


fig  1:  Benchmark  Data  Flow 


Presently,  program  Q-BENCH  runs  some  36  separate  tests 
that  measure  individual  performance  capabilities  (e.g.  disk, 
ram,  ...).  At  termination,  Q-BENCH  outputs  one  very  long  line 
of  data  which  is  appended  onto  the  database.  The  line  consists 
of  a  20-character  machine  name,  followed  by  50  blank-separated 
numeric  fields,  each  representing  performance  rates. 


Generally  speaking  there  are  two  kinds  of  benchmark  data 
one  could  measure — intervals,  and  rates.  Interval  data  is  most 
common  for  benchmarking  and  extremely  easy  to  generate  using  the 
UNIX  "time”  command.  However,  interval  data  is  hard  to  compare, 
and  suffers  from  the  complication  that  "smaller"  intervals 


14 


result  from  "better"  performances. 

Q-BENCH  exclusively  determines  "rates"  of  performance, 
where  the  work-per-second  of  some  factor  is  measured.  For 
example,  disk  performance  is  recorded  in  bytes-per-second, 
system  call  performance  is  measured  in  calls-per-second,  and  so 
forth.  These  rate  measurements  yield  to  handy  mathematical 
manipulation  as  described  later. 

We  purposely  chose  a  simple  format  for  the  database  so 
standard  UNIX  utilities  could  be  used  to  manipulate  it,  and 
so  the  database  itself  could  easily  be  carried  from  machine 
to  machine.  If  necessary,  to  transport  benchmark  data,  you 
could  hand  copy  the  numbers  off  a  CRT  and  later  use  "echo"  or 
"cat"  to  append  it  onto  the  database.  Potentially,  with  this 
approach,  you  could  benchmark  non-UNIX  machines,  provided  the 
Q-bench  code  can  be  gotten  to  compile  and  run,  but  we  have 
not  tried  yet.  Other  portability  considerations  are  discussed 
in  section  4.0  Software  Engineering. 


2.1  GENERATING  BENCHMARK  REPORTS 


The  fun  has  only  just  begun.  While  a  standard  set  of 
reports  can  be  generated,  the  usefulness  of  the  reports  depends 
upon  how  they  match  your  intended  use  for  the  systems.  Chances 
are  great  you  know  whether  RAM  performance  is  more  important  for 
your  applications,  or  FLOATing  point,  or  DISK  and  so  forth. 


view  for  mixl 

63831 

57661  1 


10000 


view  for  mix2 


20181  1 
I 

14431  1 

I 

2921  1  1 

0| - 

uchl  itch2  uch3  mcM  ucfa5  ucfa6 

■1x1  legend: 

1  <=>  float 


32241  1 

25091  1 

18541  1  1 

I 

8241  1 

0| - 

■#chl  aach2  Mch3  ucb4  »«ch5  uch6 
»ix2  legend: 

1  <=>  disk 


fig  2:  Single-factor  Projections 


The  primary  display,  called  a  VIEW,  compares  multiple- 
machine  performances  on  a  single  chart.  Two  single-factor  views 
are  show  in  figure  2.  Notice  there  is  little  consistency 
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between  floating-point  performance  and  disk. 


2.2  FORMING  COMPOUND  MIX  EXPRESSIONS 


In  keeping  with  UNIX  philosophy,  we  shunned  interactive 
parameter  entry,  and  instead  built  the  report  generator  to  read 
named  files  for  the  pre-rolled  analysis  formulas.  The  format 
is  trivial,  so  your  favorite  editor  can  be  used  to  create  new 
mix  files.  For  example,  a  mix  file  containing  just  “float", 
causes  a  report  of  those  tests  associated  with  floating-point 
performance.  Similarly,  "disk"  causes  a  report  of  just  disk 
performance  tests.  The  report  generator  internally  transforms 
each  mnemonic  mix  name  to  a  weight  vector,  by  simple  table 
lookup.  A  hidden  text  file  (defs)  contains  mnemonic  vector 
definitions.  It  too  can  be  edited. 


view  for  »ix3 

99001  1 

92031  2 

84041  3 

76061  4 

I 
I 
I 
I 
I 
I 

31921  144 

24831  4  2  3 

18351  4  1 

I 

8171  4 

01 - 

Mcfal  uch2  uch3  ucM  >ach5  iach6 

■1x3  legend: 

1  <»  Ofloat  lOOdiak 

2  <->  1 Ofloat  90disk 

3  <*>  20float  dOdisk 

4  <»  30float  70diak 


Figures  2  and  3  are  projected  fro*  the  aaae 
database  accuaulated  by  benchaarking  6  aachines 
with  qbench.  Machine  identities  have  been 
removed.  The  vertical  axis  is  serialized  on  a 
10000  point  scale,  “up”  corresponds  to  higher 
performance. 

In  figure  2,  “aixl"  compares  floating  point 
instruction  speeds,  and  -aix2"  compares  disk 
transfer  speeds.  In  figure  3,  "aix2"  considers  4 
different  application  aixtures  for  various 

of  floating  and  disk.  See  text  for 


fig  3:  Two-Factor  Projection 


Mixes  can  be  expressed  as  single  factors,  or  as  linear 
combinations  of  single  factors.  Thus  complex  analyses  can  be 
described  and  carried  out.  For  example,  you  can  devise  a  mix 
file  which  carefully  varies  a  series  of  weights  as  show  below. 


Ofloat 
lOfloat 
20f loat 
30f loat 


lOOdiak 

90disk 

80disk 

70disk 


Notice  how  the  smooth  10-percent  steps  examine  different 
floating  point  and  disk  performance  needs.  Figure  3  reveals 
several  up/down  performance  trends  that  can  be  picked  out  by 
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inspection. 

Regretably,  some  performance  marks  appear  to  be  missing 
because  the  PLOT  utility  cannot  display  overlapping  projections. 
Whenever  performances  pile  onto  the  same  PLOT  point,  only  the 
higher-numbered  label  will  be  visible.  Nevertheless,  this  lack 
of  output  resolution  usually  doesn't  hide  performance  trends. 


3.0  MATHEMATICAL  BACKGROUND 


The  main  obstacle  to  understanding  and  modelling  computer 
performance  is  the  inability  to  do  so  with  single  parameters. 
Although  human  perception  demands  it,  there  simply  is  no 
‘‘single"  figure  of  merit  which  will  suffice  all  the  time  for  all 
analyses.  Computer  performance  is  sufficiently  complicated, 
that  many  components  are  needed  to  distinguish  and  quantify 
performance . 

The  way  we  handle  this  problem  is  not  only  programmatically 
easy,  but  central  to  the  power  of  our  method.  Simply  stated, 
all  components,  <50  at  present),  are  carried  along  in  all 
necessary  calculations  but  at  the  last  moment  reduced  to  a 
single  f igure-of-merit  (“points")  by  linear  projection. 
Mathematically  speaking,  we  multiply  the  performance  data  with 
the  weight  vector  implied  from  the  mnemonic  mix. 


test 

test  test  .  .  . 

criteria 

scores 

1 

1  ,  -  - 

2  3 

weights 

■achine  1 

1 

1 

1 

1  1 

1  1  w 

1 

1 

1  1 

1  a  1 

1 

1  1  1 

1 

1  1  1 

1 

d 

1  1 

1 

1  1 

(Single 

MChiM  2 

1 

i  3 

1  1  w 

1 

1  ®  1 

1 

1  1  2 

1 

1  2  1 

Score 

1 

DATABASE 

1  X  1 

1  = 

1  1 

z 

■achine  3 

1 

1 

1  1  w 

1  1  3 

1 

i 

1  8  1 

1  3  1 

per 

a 

a 

• 

1 

1 

1 

1 

1 

1 - 

1  1  o 

1  V 

1  a 

1  1 

1 

1 

1 

1 

1 

1  V  1 

1  1 

i  1 

1  1 

1  1 

Machine 

fig  4:  Linear  Projection 


Projected  scores  from  this  method  have  several  nice 
properties.  The  most  favorable  property  is  that  of  linearity. 
Twice  the  points  corresponds  to  twice  the  performance;  charts 
based  on  “points"  can  be  directly  compared.  Furthermore, 
total-possible  points  is  easily  derived.  In  all  our  figure 
examples,  the  mnemonic  defs  file  contained  100-point  mnemonic 
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definitions  and  with  100-percent  weights  this  gives  a  final 
10000  (=100X100)  linear  point  scale.  Finally,  since  scores  are 
computed  as  linear  combinations  of  linear  combinations,  ANY 
other  analysis  process  consisting  of  linear  combinations  will  be 
equivalent  to  our  method. 


3.1  GENERALIZED  PROJECTIONS 


The  simple  linear  calculations  we  use  allow  a  surprizingly 
powerful  generalization  for  specific  application  needs.  First, 
consider  how  application  packages  can  be  parameterized  as 
weights.  For  example,  to  model  word  processing,  you  could  say 
word  processing  needs  good  RAM  performance,  some  PORT 
performance,  and  a  little  DISK  performance,  but  no  FLOATing 
point  performance.  Then,  by  treating  "application" 
definitions,  the  same  as  performance  factors,  the  same  report 
generator  program  will  perform  appl ication— or iented  evaluation 
as  well  as  factor— or iented  evaluation.  The  overhead  for  this 
flexibility  is  just  a  few  more  lines  in  the  mnemonic  definitions 
file. 


view  for  «i*4 


72691 

69401 

I 

56281 

I 

I 

44161 

38411 

I 

31401 

24761 

21401 

14641 

7381 

I 

01 


3 

2 

1 


3 


iachl  uch2 
■ix4  legend: 

1  <=>  lOus  90sp 

2  <=>  50co  50*p 

3  <=>  100m: 


1 

2 

3  3 
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fig  5:  Appplication-Or iented  Projection 


Figure  5  shows  a  complicated  what-if  projection  for  three 
mixes  running  on  the  same  machines  from  figures  2  and  3.  It 
shows  how  the  systems  should  perform  for  three  application- 
oriented  loads.  Recall  that  the  view  is  really  just  a 
mathematical  projection,  not  an  actual  performance  measurement. 
The  first  mixture  line  means  ten-percent  userfriendly  services 
and  ninety— percent  spreadsheet,  the  second  mix  line  means 
f ifty— percent  compiling  and  f if ty— percent  spreadsheet,  and  the 
final  mix  line  means  one  hundred-percent  accounting  use.  Values 
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for  the  exact  weights  defined  as  userfriendly  services  (us), 
spreadsheet  <sp) ,  and  so  forth,  are  listed  in  the  appendix. 
(They  were  voted  upon  by  a  committee  of  benchmark  experts 
(friends)  and  left  intentionally  easy  to  edit  to  reflect 
personal  tastes.) 

Theoretically,  there  are  an  infinite  number  of  ways  to 
quantify  application-oriented  needs.  Any  two  committees  need 
not  agree  on  the  particular  weights  in  question,  but  if  they 
agree  to  use  linear  modelling,  they  can  both  use  the  same  report 
generator.  Our  reporting  method,  which  is  subjective  in  just 
this  one  respect,  nevertheless  seems  to  produce  consistent 
results.  All  other  portions  of  our  benchmarking  and  report 
generation  method  are  objective,  and  mathematically  justifiable. 


4.0  SOFTWARE  ENGINEERING 


In  retrospect,  we  can  see  how  design  decisions  were 
influenced  by  the  nature  of  UNIX.  In  shopping-list  form,  we 
took  full  advantage  of  many  UNIX  and  C  features:  We  used 
#define  symbols  to  control  different  OS  versions,  #include  files 
and  V7  system  calls  for  measurements,  "make"  to  encapsulate 
benchmarking  and  reporting  procedures,  and  shell  scripts  to 
remember  option  settings. 

Certainly  the  Bourne  shell  had  much  impact,  suggesting  if 
you  will,  a  preferred  order  of  implementation.  A a  shown  in 
figure  6,  there  are  three  layers  to  be  seen  in  the  project 
software:  Specialized  C-code,  general-purpose  C-code,  and 
specialized  shellscripts. 


SPECIAL-PURPOSE 

C-PROGRAHS 


1170  qbencb.c 
760  select.c 


GEXERAL-PURPOSE 

C-PROGRAHS 


138  cksua.c 
563  plot.c 


1949  lines 


699  lines 


order  of  iiplesentation  — > 


SHELL-FILES  and 
SCRIPTS 


15  KIVIAT 
15  POIMTS 

23  VIEW 
46  defs 

24  sakefile 
7  aixl 

1  aix2 
3  «ix3 
■ix4 
■ix5 
■ix6 
■ix8 
■ix9 


150  lines 


fig  6:  Software  Design  Layers 
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Following  the  "fail-early"  principle  (John  Mashey-"Small  ia 
Beautiful"  talk  Dec  1982),  we  atarted  coding  the  hardeat 
programs  firat.  The  reaaoning  being  it  ia  more  preferable  to 
fail  early  than  to  fail  late  in  a  project.  Secondly,  in  the 
event  we  ran  out  of  time  to  develop  everything  aa  planned,  we 
could  akimp  on  the  general  C-code,  and  akip  some  ahellscripts, 
but  still  have  a  working  product. 

Our  final  observation  concerns  the  number  of  text  lines 
within  each  layer.  Each  successive  layer  uses  fewer  lines. 
Presumably  this  is  because  each  higher  layer  utilizes  more 
"general"  (and  presumably  familiar)  programming  concepta.  Each 
layer  certainly  was  easier  to  code,  and  it  was  moat  pleasant  not 
to  deal  with  prematurely-closed  design  decisions.  It  is  all  too 
common  for  the  10-percent  remainder  of  a  programming  project  to 
consume  equally  as  much  work  as  the  first  90  percent.  In  our 
case,  the  final  layer  took  about  2  days  to  develop  and  perfect, 
while  the  first  layer  took  about  4  man-months.  The  middle  layer 
programs  were  actually  borrowed  from  another  project. 


4.1  SOFTWARE  PORTABILITY  ISSUES 


The  need  for  portability  in  benchmarking  packages  goes 
without  saying.  Benchmarks  should  run  easily  on  a  lot  of 
different  systems  in  order  for  them  to  have  any  chance  of  being 
used.  In  the  name  of  "portability"  we  coded  conservatively,  and 
wrote  all  intermediate  files  in  clear  ascii  text.  The  implicit 
relationships  in  our  design  from  separating 

mix-files  and  defs-files  from  the  benchmark  database  encouraged 
conceptual  portability  as  well. 

Nevertheless,  we  ran  into  a  few  difficulties. 

*  clock/tick  dependence 

50Hz,  60Hz  clocks  and  100:1  cpu  range 

*  16-bit  integer  precision 

int  x  ->  unsigned  x 

*  compiler  bugs 

aCa[a[a[a[a[a[a[a[a(a[aCa[a[a[a[i]] ]]]]]]]] ]]}]]]  -> 

a  [a  [a  C a  C i 3  ]  ]  ]  ,  a  [a  Ca  [a  Cil  ]  ]  ] ,  a  ta  [a  Ca  ti]  ]  3 1 ,  a  [a  [a  [a  til  1  ]  ] 

*  crt/lineprinter  resolution 

crt  80X24  and  linepr inter  66X132 

In  order  to  sidestep  a  multitude  of  known  weaknesses  with 
UNIX  clocks,  we  devised  an  auto-ranging  strategy  for  taking 
measurements.  The  auto-ranging  function  runs  each  benchmark 
until  a  minimum  reliable  interval  is  exceeded.  Cpu-tests  are 
run  at  least  2  seconds.  Disk  and  port  tests  run  up  to  30  seconds 
as  we  found  this  necessary  to  void  buffer  caching.  To  further 
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increase  reliablity,  three  distinct  runs  of  the  benchmark  are 
averaged  together  internally.  We  found  averaging  three  runs 
provides  sufficient  smothing  effect  for  clock  "noise". 
Actually,  to  top  if  off,  we  usually  suggest  running  Q-bench 
twice  on  each  machine,  just  to  be  sure. 

We  ran  into  difficulties  on  some  very  fast  16-bit  machines, 
where  the  RAM  tests  needed  all  16-bits  to  represent  loop  counts. 
Installing  a  few  "unsigned"  declarations  fixed  this  problem. 
More  treacherous  was  the  16-deep  subscripting  test,  which  made 
several  compilers  loop  endlessly,  and  generate  bad  code.  We 
simplified  that  test  in  the  name  of  portability. 

Finally,  we  made  a  few  concessions  for  portability  when 
designing  report  formats.  We  implemented  "height"  and  "width" 
parameters  in  the  report  generator  and  set  them  via  shellscripts 
so  the  user  is  unbothered  by  such  details.  This  way,  the 
scripts  adapt  to  whether  output  is  for  CRT  or  lineprinter.  Even 
so,  some  reports  simply  will  never  fit  on  a  crt. 


5.0  CONCLUSIONS  AND  COMMENTS 


With  respect  to  software  design,  we  make  two  observations. 
First,  separating  benchmarking  activities  and  reporting 
activities  was  clearly  a  win.  Using  a  database  to  connect  the 
activities  created  the  possibility  of  using  other  analysis 
programs  besides  our  own.  Conversely,  the  report  generator  does 
not  depend  on  Q-bench  for  its  data,  but  can  read  any  column- 
oriented  data  file.  Thus  VIEW  projections  can  be  used  for 
evaluating  most  any  performance— related  data. 

Second,  good  software  doesn't  HAVE  to  prompt.  We  think  this 
project  goes  a  long  way  to  simplifying  benchmarks  and  yet  is 
straightforward  to  use. 

Furthermore,  pleasant  surprizes  can  happen,  even  in  fairly 
cut  and  dried  benchmark  software  projects.  Several  people  have 
remarked  Kiviat  diagrams  approach  "art"  in  their  own  right.  (Of 
course,  we  feel  they  convey  somewhat  more  information  than  just 
that.)  Kiviat  diagrams  were  not  even  supposed  to  "work"  on 
line-printer  plots,  as  printer  resolution  was  deemed  inadequate. 

Pleasant  diversions  can  also  happen,  even  in  cut  and  dried 
projects.  We  will  only  issue  the  challenge  here,  the  solution 
will  be  discussed  at  the  talk. 

"CHECKSUMMED  HEADER"  PUZZLE 
It  is  deemed  essential  to  put  a  small  header 
file  at  the  head  of  a  certain  unnamed 
company's  software  packages.  The  header 
file  must  say  something  to  the  effect,  "/* 
serial  no.  XXX,  checksum=YYY  */". 
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Furthermore,  if  anyone  checksums  such  a 
program  file  (including  its  header)  it  must 
match  the  value  YYY  as  announced  in  its 
header.  How  would  you  create  the  header 
files? 

Finally,  we  must  conclude  whether  or  not  the  project  goals 
were  met.  One  goal  was  to  automate  the  benchmarking  process  for 
non-experts.  We  believe  we  succeeded.  Non-experts  can  compare 
machine  performance  in  terms  of  ’•word-processing",  "data-base 
operations",  "compiling",  etc.  with  as  much  relevance  as  experts 
can  compare  "disk-thruput" ,  "divide-times",  and  so  forth. 

Practically  speaking,  we  have  only  begun  to  unfold  these 
methods.  Because  they  cleanly  organize  and  report  benchmark 
data  in  a  non-expert  fashion,  we  believe  they  will  quickly 
become  a  useful  standard  for  benchmark  presentation. 

We  want  to  thank  many  people  for  help  during  this  project, 
it  could  not  have  happened  otherwise.  Special  thanks  goes  to 
Walter  Zintz  who  provided  steadfast  support  and  insisted  "disk- 
thruput"  is  NOT  user-friendly,  Kirk  McKusic,  who  suggested  even 
more  difficult  problems.  Together  these  two  inspired  the  linear 
modelling  idea  so  crucial  to  the  project.  Thanks  to  Bill  King 
who  introduced  Kiviat  diagramming,  Linda  Nied  and  Breck  Beatie 
who  provided  much  appreciated  moral  support  and  feedback  during 
development.  Finally,  we  thank  Jim  Geers  and  Aim  Technology  for 
funding  this  research. 


S.O  APPENDICES 

1.  A  Kiviat  Diagram 

2.  Sample  Q-bench  log 

3.  Sample  benchmark  output 

4.  Mnemonic  "defs"  file 
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SAMPLE  Q-BENCH  RUNNING  COMMENTARY 

What  i«  this  aachine's  nue:  labrat 
What  la  ita  price  in  dollara:  29000 

Teating  Started. 


TE3T00  labrat  29000. 
TEST01  getpidO  calls  1824: 
TEST02  sbrk(0)  calls  796: 
TEST09  creat-cloae  calls  25: 
TEST39  uaask(0)  calls  1480: 
TEST04  process  forks  5: 
TEST05  ttyl  write  696: 
TEST36  ttyl+tty2  write  787: 
TEST06  disk  write  19765: 
TEST07  disk  read  22016: 
TEST08  disk  copy  9506: 
TEST09  pipe  copy  70095: 
TEST10  add  LONG  640000: 
TEST11  add  SHORT  205998: 
TEST12  add  FLOAT  1679: 
TEST94  add  DOUBLE  1684: 
TEST19  array  ref  short [short!  129491: 
TEST14  array  ref  longlloog!  140502: 
TEST15  call  functO  48852: 
TEST16  call  funct(int)  99909: 
TEST17  call  funct(int,int)  26852: 
TEST18  rai  read  SORT  1170759: 
TEST19  raa  read  LONG  1662694: 
TEST20  ru  read  CHAR  599068: 
TEST21  ru  write  SHORT  702452: 
TEST22  ru  write  LONG  960018: 
TEST23  ru  write  CHAR  353932: 
TEST24  ru  copy  SHORT  797932: 
TEST25  ru  copy  LONG  990988: 
TEST26  ru  copy  CHAR  404216: 
TEST27  aultiply  SHORT  20540: 
TEST28  aultiply  LONG  19640: 
TEST29  aultiply  FLOAT  1759: 
TEST35  aultiply  DOUBLE  1700: 
TEST30  divide  SHORT  12891: 
TEST31  divide  LONG  11218: 
TEST32  divide  FLOAT  931: 
TEST37  divide  DOUBLE  931: 
Testing  Coapleted. 


2  kcalls/sec  or  548  aicrosec/call 
796  calls/sec  or  1256  aicrosec/call 
25  palrs/sec  or  40000  aicrosec/pair 

1  kcalls/sec  or  676  aicrosec/call 
5  forks/sec  or  200000  aicrosec/fork 

696  bytes/sec  or  1437  aicrosec/byte 
787  bytes/sec  or  1271  aicrosec/byte 

20  kbytes/sec  or  51  aicrosec/byte 
22  kbytes/sec  or  45  aicrosec/byte 

10  kbytes/sec  or  105  aicrosec/byte 
70  kbytes/aec  or  14  aicrosec/byte 

640  kadds/sec  or  2  aicrosec/add 
205  kadds/sec  or  5  aicrosec/add 

2  kadds/sec  or  598  aicrosec/add 
2  kadds/sec  or  594  aicrosec/add 

129  krefs/sec  or  8  aicrosec/ref 
141  krefa/sec  or  7  aicrosec/ref 
49  kcalls/sec  or  20  aicrosec/call 
94  kcalls/sec  or  29  aicrosec/call 
27  kcalls/sec  or  37  aicrosec/call 
1171  kbytes/sec  or  854  nanoaec/byte 
1663  kbytes/sec  or  601  nanoaec/byte 
593  kbytes /sec  or  2  aicrosec/byte 
702  kbytea/sec  or  1  aicrosec/byte 
960  kbytes/sec  or  1  aicrosec/byte 
354  kbytes/sec  or  3  aicrosec/byte 
798  kbytea/sec  or  1  aicrosec/byte 
991  kbytes/sec  or  1  aicrosec/byte 
404  kbytes/sec  or  2  aicrosec/byte 

21  kaults/sec  or  49  aicrosec/ault 
20  kaults/sec  or  51  aicrosec/ault 

2  kaults/sec  or  569  aicrosec/ault 
2  kaults/sec  or  588  aicrosec/ault 
13  kdivs/sec  or  78  aicrosec/div 

11  kdivs/sec  or  89  aicrosec/div 
931  divs/aec  or  1074  aicrosec/div 
931  divs/aec  or  1074  aicrosec/div 
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SAMPLE  MACHINE  REPORT 


ARITHMETIC  IMSTRDCTIOi  TIMES  (aicroaecooda  per  op) 
abort  long  float  double 


+  add 

5 

2 

598  594 

•  aultiply 

49 

51 

569  588 

/  divide 

78 

89 

1074  1074 

MEMORY  LOOP  ACCESS  TINES 

(aicroaecooda  per  byte) 

read 

write 

copy 

CHAR  type 

2 

3 

2 

SHORT  type 

854ns 

1 

1 

LOME  type 

601ns 

1 

1 

IKPUT/OUTPUT  RATES  (bytea/aec) 

read 

write 

copy 

DISK 

22k 

20k 

10k 

PIPE 

70k 

TTY  1 

696 

TTY  1+2 

787 

RAH  1-byte 

404k 

RAM  4-byte 

991k 

ARRAY  SUBSCRIPT  REFERENCES  (aicroaecooda) 
abort  U  loogU 

8  7 


FUNCTION  REFERENCES  (aicroaecooda/ref) 


0-par aietera 
functO 
20 

PROCESS  FORKS 
(  39k  bytea) 
5  per  second 


1-par  Meter 
funct(i) 

29 


2-par aietera 
funct(i,i) 
37 


SYSTEM  KERNEL  CALLS 
getpidO  calls: 
abrk(0)  calls: 
create/close  calls: 
umaak(O)  calls:  1 


(calla-per-second  and  aicroaecooda  per  call) 
2  kcalls/aec  or  548  aicroaeconds/call 
796  calla/aec  or  1256  aicroaeconds/call 
25  paira/aec  or  40000  aicroaeconds/pair 
kcalls/aec  or  676  aicroaeconds/call 
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all 

all 

all 

all 

all 

all 

all 

all 

all 

all 

all 

all 

all 

all 

all 

all 

all 

all 

all 

all 

all 

ram 

math 

float: 

disk 

tty 

pipe 
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wp 

wp 

db 

db 

ap 

ap 

ac 

ac 

ac 

ac 

co 

co 

co 

gr 

gr 

ua 

ua 

ua 


MNEMONIC  DEFINITIONS  FILE 

all  50  testa  froa  qbench 
1, l,getpid(),sy seal 1, sec  l,2,Bbrk(),syscall,sec 
l,3,creat-clo,file,sec  1, 4, forkO, foresee 

l,5,ttyl,byte,sec  1,6, diskjirt, byte, sec 

l,7,disk_rd,byte,sec  l,8,disk_cpy, byte, sec 

1,9, pipe_cpy, byte, sec  l,10,add_long,add,sec 

l,ll,add_short,add,sec  l,12,add_float,add,aec 
1,13,  shortU,  ref,  sec  l,14,long[],ref,aec 

l,15,funct(),call,aec  1,16, funct(i), call, sec 

l,17,funct(2i),call,sec  l,18,raa_rshort,byte,sec 
l,19,raa_rlong, byte, sec  l,20,raa_rchar,byte,sec 
l,21,raa_wahort,byte,sec  l,22,raa_wlong,byte,sec 
l,23,ram_vcbar,byte,aec  l,24,raa_csbort,byte,sec 
l,25,rai_cloag,byte,sec  l,26,raa_cchar,byte,sec 
l,27,aul_sbort,ault,sec  l,28,aul_long,ault,sec 
l,29,aul_float,ault,sec  l,30,div_short,div,sec 
l,31,div_loog,div,sec  l,32,div_float,div,sec 

1.33, uaask(),call,sec 

1.34, add_double,add,sec  l,35,aul_double,ault,sec 

1.36,  ttyl+tty2 , byte, sec 

1.37, div_double,div,sec 

10.20  10,18  10,19  10,20  10,21  10,22  10,23  10,24  10,25  10,26  -raa 

16,27  16,28  20,11  16,10  16,30  16,31  -  math  =  integer  +  »  / 

20,12  16,29  16,32  16,34  16,35  16,37  -  float  =  flt+dbl  +  »  / 

25,03  25,06  25,07  25,08  -  disk  =  cre/clo+r/w/copy 

51,05  49,36  -  ttys  =  one  and  two-write 

100,09  -  pipe  =  pipe  copy 

34,15  33,16  33,17  -  logic  =  function  calls 

word-processing  -  Such  string  raa,  disk  and  tty. 

32.20  1,15  1,27  1,29  32,8  1,3  1,9  32,5 

data-base  operations  -  Huch  raa,  logic,  disk,  and  file  ops. 

24.20  24,15  1,27  1,29  24,8  24,3  1,9  1,5 

spread-sheet  operations  -  Much  raa,  logic,  aath  and  tty. 

1,20  24,15  24,27  24,29  1,8  1,3  1,9  24,5 

scientific  calculations  -  Huch  raa,  logic,  aath,  disk  and  tty. 

1,20  24,15  24,27  24,29  24,8  1,3  1,9  1,5 

accounting  operations  -  Huch  logic  and  aath. 

1,20  19,15  19,27  19,29  21,8  19,3  1,9  1,5 

compiling  and  language  developaent  -  Huch  logic,  integer  aath, 
disk,  and  file-ops. 

1,20  24,15  24,27  1,29  24,8  24,3  1,9  1,5 

graphics  operations  -  Huch  raa,  logic,  integer  aath. 

32.20  32,15  32,27  1,29  1,8  1,3  1,9  1,5 

user-friendly  interface  operations  -  Huch  raa,  file-ops,  pipes, 
and  tty. 

24.20  1,15  1,27  1,29  1,8  24,3  24,9  24,5 


